Disco Diffusion Batch Generation

Note

Install ekorpkit package first.

Set logging level to Warning, if you don’t want to see verbose logging.

If you run this notebook in Colab, set Hardware accelerator to GPU.

!pip install –upgrade –pre ekorpkit[disco] exit()

Prepare Environment

from ekorpkit import eKonf

eKonf.setLogger("WARNING")
eKonf.set_cuda(device=4)
print("version:", eKonf.__version__)

is_notebook = eKonf.is_notebook()
is_colab = eKonf.is_colab()
print("is notebook?", is_notebook)
print("is colab?", is_colab)
if is_colab:
    eKonf.mount_google_drive()
eKonf.set_workspace(workspace="/content/drive/MyDrive/workspace/", project="ekorpkit-book")

INFO:ekorpkit.base:Setting cuda device to ['A100-SXM4-40GB (id:0)']
INFO:ekorpkit.base:Google Colab not detected.

version: 0.1.38+17.g2028903.dirty
is notebook? True
is colab? False

Create a disco instance

cfg = eKonf.compose("model/disco")
disco = eKonf.instantiate(cfg)

INFO:ekorpkit.base:Loaded .env from /workspace/projects/ekorpkit-book/config/.env
INFO:ekorpkit.base:setting environment variable CACHED_PATH_CACHE_ROOT to /workspace/.cache/cached_path
INFO:ekorpkit.base:setting environment variable KMP_DUPLICATE_LIB_OK to TRUE

Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off]
Loading model from: /opt/conda/lib/python3.8/site-packages/lpips/weights/v0.1/vgg.pth

Batch generation

batch_name = "newworld"
text_prompts = "Brave and beautiful new world in Jeju, artstation matte painting."
set_seed = 123
batch_args = {
    "clip_guidance_scale": [1000, 5000, 10000, 50000],
    "steps": [50, 100, 250, 500],
    "tv_scale": [0, 1000, 10000, 20000],
    "eta": [0.0, 0.4, 0.8, 1.0],
    "clamp_max": [0.01, 0.05, 0.5],
    "cut_overview": ["[12]*1000", "[12]*400+[4]*600", "[1]*1000"],
    "cut_innercut": ["[12]*1000", "[4]*400+[12]*600", "[1]*1000"],
    "cut_ic_pow": [1, 10, 100],
    "cutn_batches": [2, 4, 8],
}
batch_args = {
    "clip_guidance_scale": [1000, 5000],
    "steps": [50, 100],
    "tv_scale": [0, 1000],
    "eta": [0.0, 1.0],
    "clamp_max": [0.01, 0.5],
    "cut_overview": ["[12]*1000", "[1]*1000"],
    "cut_innercut": ["[12]*1000", "[1]*1000"],
    "cut_ic_pow": [1, 100],
    "cutn_batches": [1, 4],
}
batch_pairs = [
#     ("clip_guidance_scale", "steps", "tv_scale"),
    ("eta", "steps"),
#     ("clamp_max", "steps"),
#     ("eta", "clamp_max"),
#     ("cut_overview", "cut_innercut"),
#     ("cut_ic_pow", "cutn_batches"),
]
batch_results = disco.batch_imagine(
    text_prompts=text_prompts, 
    batch_name=batch_name, 
    batch_args=batch_args, 
    batch_pairs=batch_pairs,
    set_seed=set_seed
)
print(batch_results)

1 samples generated to /workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/newworld_eta_steps
text prompts: Brave and beautiful new world in Jeju, artstation matte painting.
sample image paths:
/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/newworld_eta_steps/newworld_eta_steps(23)_0000.png

Saved collage to /workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_eta_steps_batch_configs.png
['/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_eta_steps_batch_configs.yaml']

# batch_results = ['/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_clip_guidance_scale_steps_tv_scale_batch_configs.yaml', '/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_eta_steps_batch_configs.yaml', '/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_clamp_max_steps_batch_configs.yaml', '/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_eta_clamp_max_batch_configs.yaml', '/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_cut_overview_cut_innercut_batch_configs.yaml', '/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_cut_ic_pow_cutn_batches_batch_configs.yaml']
batch_results

['/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_clip_guidance_scale_steps_tv_scale_batch_configs.yaml',
 '/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_eta_steps_batch_configs.yaml',
 '/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_clamp_max_steps_batch_configs.yaml',
 '/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_eta_clamp_max_batch_configs.yaml',
 '/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_cut_overview_cut_innercut_batch_configs.yaml',
 '/workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_cut_ic_pow_cutn_batches_batch_configs.yaml']

Collage batch results

for batch_config_path in batch_results[3:5]:
    disco.batch_collage(
        batch_config_path=batch_config_path,
        prompt_fontsize=30,
        fontsize=30,
    )

Saved collage to /workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_eta_clamp_max_batch_configs.png

Saved collage to /workspace/projects/ekorpkit-book/disco-imagen/outputs/disco-diffusion/batch_configs/newworld_cut_overview_cut_innercut_batch_configs.png